There has been a technique for compositing images photographed with a plurality of cameras mounted around a vehicle to generate a bird's-eye view of the vehicle as viewed from above the vehicle.
For example, according to Japanese Patent No. 3286306, a bird's-eye view is generated by mapping camera images on a plane assuming that the entire objects photographed with cameras exist on a road surface. FIG. 17 is a diagram depicting an example of the bird's-eye view formed based on Japanese Patent No. 3286306. Japanese Patent No. 3286306, however, has a problem that a stereoscopic object having a height component above the road surface is distorted largely as depicted in FIG. 17. This problem is solved in, for example, Japanese Laid-open Patent Publication No. 2003-132349 or Japanese Laid-open Patent Publication No. 07-17328.
In Japanese Laid-open Patent Publication No. 2003-132349, an elliptical stereoscopic projection plane around a vehicle is set and camera images are projected through linear mapping on the elliptical stereoscopic projection plane, so that a bird's-eye view is generated. FIG. 18 is a diagram depicting an example of a bird's-eye view formed based on Japanese Laid-open Patent Publication No. 2003-132349. As depicted in FIG. 18, since the stereoscopic projection plane has a height component, the distortion of the stereoscopic object can be suppressed as compared with the distortion in Japanese Patent No. 3286306.
In Japanese Laid-open Patent Publication No. 07-17328, the shape of the stereoscopic object around a vehicle is measured using a range sensor or the like. Then, in Japanese Laid-open Patent Publication No. 07-17328, based on the measurement results, a partial projection plane having a height component is installed at a position where the stereoscopic object actually exists, and camera images are nonlinearly projected on this projection plane, so that a bird's-eye view is generated. FIG. 19 is a diagram depicting an example of the bird's-eye view formed based on Japanese Laid-open Patent Publication No. 07-17328. As depicted in FIG. 19, since the camera images are projected to the projection plane having a height component, the distortion of the stereoscopic object can be suppressed.
Patent Document 4: Japanese Laid-open Patent Publication No. 2002-359838
Patent Document 5: Japanese Laid-open Patent Publication No. 2009-298178
Patent Document 6: Japanese Laid-open Patent Publication No. 2006-253872
In the patent documents described above, however, there has been a problem that the stereoscopic object is not displayed in bird's-eye view as appropriate.
For example, in Japanese Laid-open Patent Publication No. 2003-132349, since the camera images are projected to the stereoscopic projection plane through linear mapping, the stereoscopic object may be mapped outside the stereoscopic projection plane depending on the position of the stereoscopic object. FIG. 20 is a diagram for describing the problem of the conventional technique. In an actual space 1 to be photographed by cameras where persons 1a to 1g are present, the persons 1c to 1g included in a range 2b are not mapped on a stereoscopic projection plane 2a in a mapped virtual space 2, so that the persons 1c to 1g get out of the range of the stereoscopic projection plane 2a. FIG. 18 also illustrates the stereoscopic object in a region 3, which is out of the stereoscopic projection plane.
Moreover, although Japanese Laid-open Patent Publication No. 07-17328 discloses the installation of a partial projection plane, how the camera images are projected to this projection plane is not disclosed; therefore, it cannot be said that the stereoscopic object is displayed in a bird's-eye view with high accuracy.